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Lessons  Learned  From  Acquisitions  of 
Modified  Commercial  Items  and  Nondevelopmental  Items 


Executive  Summary 

Introduction.  This  report  discusses  buying  organizations’  lessons  learned  from  the 
acquisitions  of  modified  commercial  items  and  nondevelopmental  items.  For  more  than 
20  years,  DoD  policy  has  been  to  acquire  suitable  commercial  and  nondevelopmental 
items  instead  of  developing  new  items.  Enactments  of  the  Federal  Acquisition 
Streamlining  Act  of  1994  and  the  Federal  Acquisition  Reform  Act  of  1996  broadened 
the  definitions  of  commercial  items  and  made  it  easier  for  buying  organizations  to  use 
streamlined  acquisition  procedures  for  acquiring  modified  commercial  and 
nondevelopmental  items.  In  addition,  DoD  Regulation  5000. 2-R,  “Mandatory 
Procedures  for  Major  Defense  Acquisition  Programs  (MDAPs)  and  Major  Automated 
Information  System  (MAIS)  Acquisition  Programs,”  March  15,  1996,  directs  buying 
organizations  to  define  requirements  in  terms  that  enable  and  encourage  commercial 
offerors  to  compete  to  fill  such  requirements. 

Audit  Objective.  The  audit  objective  was  to  determine  lessons  learned  from  the 
acquisition  management  of  Defense  systems  developed  and  procured  using  modified 
commercial  item  and  nondevelopmental  item  acquisition  strategies.  The  audit  also 
evaluated  the  effectiveness  of  management  controls  related  to  Ae  objective. 

Audit  Results.  Defense  buying  organizations  have  made  progress  in  strengthening 
acquisition  strategies  used  to  acquire  modified  commercial  and  nondevelopmental 
items.  The  buying  organizations  identified  lessons  learned  that  described  efforts  taken 
to  manage  program  uncertainties  resulting  from  the  transfer  of  acquisition  management 
control  from  buying  organizations  to  commercial  suppliers.  Program  management 
efforts  intensified  because  to  remain  competitive,  commercial  suppliers  often  retain 
proprietary  rights  to  technical  data  that  often  affect  or  describe  product  pCTformance, 
quality,  and  logistical  support.  To  effectively  manage  modified  commercial  Md 
nondevelopmental  items,  buying  organizations  must  identify  program  uncertainties 
applicable  to  the  acquisitions  and  develop  acquisition  strategies  to  manage  the  program 
uncertainties. 

The  management  controls  were  effective  in  that  no  material  management  control 
weakness  was  identified.  See  Appendix  A  for  details  on  the  management  control 
program. 

Management  Comments.  We  provided  a  draft  of  this  report  on  July  18,  1997. 
Because  this  report  contains  no  findings  or  recommendations,  written  comments  were 
not  required,  and  none  were  received.  Therefore,  we  are  publishing  this  report  in  final 
form. 
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Part  I  -  Audit  Results 


I 


Audit  Background 


DoD  first  emphasized  acquisitions  of  commercial  products  and  nondevelopmental 
items  when  it  created  the  Acquisition  of  Commercial  Products  Program  in  1976. 
Congress  strengthened  that  policy  by  requiring  DoD  to  give  preference  to 
acquiring  commercial  and  nondevelopmental  items  to  satisfy  military  requirements 
with  the  enactment  of  the  National  Defense  Authorization  Act  of  1986.  DoD 
further  advanced  commercial  and  nondevelopmental  item  acquisitions  by  endorsing 
open  system  architecture  designs  for  electronic  and  communication  systems, 
subsystems,  and  major  components  and  by  conducting  congressionally  directed 
foreign  comparative  tests  for  weapon  systems.  By  taking  those  actions,  DoD  can 
benefit  fi’om  technology  advancements  that  the  electronics  and  communication 
industries  made  and  from  the  acquisition  of  weapon  systems  that  foreign 
governments  developed. 

Broadened  Definition  of  Commercial  Items.  The  Federal  Acquisition 
Streamlining  Act  of  1994  and  the  Federal  Acquisition  Reform  Act  of  1996 
broadened  the  definition  of  a  commercial  item.  As  redefined,  systems,  subsystems, 
and  components  other  than  real  property  are  considered  commercial  items  or 
modified  commercial  items  if  they  are  of  a  type  that  has; 

•  been  offered  for  sale,  sold,  leased,  or  licensed  to  the  general  public; 

•  evolved  through  technology  or  performance  advances  and  will  be 
available  in  the  commercial  marketplace  in  time  to  meet  proposed  Government 
delivery  requirements;  or 

•  required  modifications  of  a  type  customarily  available  in  the 
commercial  markeqilace  or  required  minor  modifications  of  a  type  not 
customarily  available  in  the  commercial  marketplace  to  meet  DoD  requirements. 

Besides  broadening  the  definition  of  commercial  items,  the  1994  and  1996  Acts 
enabled  DoD  to  revise  and  modify  acquisition  regulations  as  follows; 

•  simplify  contracting  procedures  for  acquisitions  not  costing  more  than 
$5  million,  when  contracting  officers  reasonably  believe  that  only  commercial 
items  will  be  offered; 

•  exempt  the  contract  requirement  to  provide  cost  or  pricing  data  for 
commercial  items; 

•  give  contracting  officers  flexibility  in  detennining  price 
reasonableness  for  contracts,  subcontracts,  and  modifications; 

•  provide  contracting  officers  a  list  of  Federal  Acquisition  Regulation 
(FAR)  provisions  that  are  not  applicable  for  commercially  available  items;  and 

•  remove  more  than  30  contractor  certification  requirements  that  are  not 
specifically  imposed  by  statute. 
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The  1994  and  1996  Acts  made  it  easier  for  buying  organizations  to  acquire 
modified  commercial  items  by  eliminating  barriers  to  fully  implementing  DoD 
policy  of  acquiring  commercial  and  nondevelopmental  items  in  preference  to 
initiating  a  new  development  effort.  The  1994  and  1996  Acts  also  required 
acquisition  personnel  to  use  new  techniques  and  new  skills  for  buying 
commercial  items  to  meet  DoD  needs.  In  his  comments  to  the  Section  800 
Panel  Report*  that  led  to  the  acquisition  reforms  made  in  the  1994  and  1996 
Acts,  the  Deputy  Inspector  General,  DoD,  stated  that  commercial  suppliers 
would  not  have  to  demonstrate  viability  of  evolved  and  modified  commercial 
items  as  stand-alone  products  in  the  competitive  marketplace.  Further,  he 
emphasized  that  contracting  officers  would  not  have  an  established  baseline  to 
determine  price  reasonableness  for  evolved  and  modified  commercial  items, 
which  would  lead  to  additional  program  risk.  Accordingly,  the  1994  and  1996 
Acts  also  increased  program  risks  associated  with  buying  commercial  items. 

Definition  of  Nondevelopmental  Items.  The  1994  Act,  the  FAR,  and 
DoD  Regulation  5000.2-R,  “Mandatory  Procedures  for  Major  Defense 
Acquisition  Programs  (MDAPs)  and  Major  Automated  Information  System 
(MAIS)  Acquisition  Programs,”  March  15,  1996,  require  contracting  officers  to 
give  preference  to  procuring  commercial  items  first  and  nondevelopmental  items 
second  before  considering  the  development  of  a  new  item  to  satisfy  stated  DoD 
requirements.  The  guidance  defines  systems,  subsystems,  and  components  as 
nondevelopmental  items  if  they: 

•  were  previously  developed  exclusively  for  governmental  purposes  by 
a  Federal  agency,  a  State  or  local  government,  or  a  foreign  government  widi 
which  the  United  States  has  a  mutual  defense  cooperation  agreement, 

•  satisfy  the  previous  criteria  but  need  only  minor  modifications  to 
meet  Government  requirements,  or 

•  are  being  produced  for  governmental  purposes  but  are  not  yet  in  use. 


In  Section  800  of  Public  Law  101-510  (the  National  Defense  Authorization  Act  of  FY  1991), 
Congress  directed  DoD  to  establish  flie  “DoD  Advisory  Panel  on  Streamlining  and  Codifying 
Acquisition  Law.”  The  Advisory  Panel’s  January  1993  report,  “Streamlinmg  Defense 
Acquisition  Law,”  is  commonly  referred  to  as  the  Section  800  Panel  Report. 
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Audit  Objective 


The  audit  objective  was  to  determine  lessons  learned  from  the  acquisition 
management  of  Defense  systems  developed  and  procured  using  modified 
commercial  item  and  nondevelopmental  item  acquisition  strategies. 

Specifically,  we  reviewed  and  identified  lessons  learned  for  program  definitions; 
program  structures;  program  designs;  contracting;  program  assessments;  and 
decisions,  reviews,  and  periodic  reporting  and  management  controls  related  to 
the  objective.  In  Appendix  A,  we  discuss  the  scope  and  methodology  used  to 
accomplish  the  objective  as  well  as  management  controls.  In  Appendix  B,  we 
discuss  prior  audit  coverage.  In  Appendix  C,  we  list  the  37  modified 
commercial  and  nondevelopmental  item  acquisitions  reviewed. 
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Lessons  Learned  From  Acquiring 
Modified  Commercial  and 
Nondevelopmental  Items 

Defense  buying  organizations  have  made  progress  in  strengthening 
acquisition  strategies  used  to  acquire  modified  commercial  and 
nondevelopmental  items.  The  organizations  identified  many  lessons 
learned  from  acquiring  the  commercial-type  items  that  can  be 
summarized  as  a  shift  in  efforts  taken  to  manage  program  uncertainties 
associated  with  those  acquisitions.  For  the  most  part,  the  organizations 
identified  program  uncertainties  resulting  from  the  transfer  of  acquisition 
management  control  from  buying  organizations  to  commercial  suppliers. 
Program  management  efforts  intensified  because  to  remain  competitive 
in  the  commercial  market  place,  commercial  suppliers  often  retain 
proprietary  rights  to  technical  data  affecting  or  describing  product 
performance,  quality,  and  logistical  support.  To  effectively  manage 
modified  commercial  and  nondevelopmental  items,  the  buying 
organizations  must  identify  those  uncertainties  and  develop  acquisition 
strategies  to  manage  them,  including  r^uiring  comniercial  suppliers  to 
substantiate  product  performance,  quality,  and  logistical  support  claims 
before  the  Government  acquires  Ae  items. 


Lessons  Learned 


In  contrast  to  new  development  efforts,  buying  organizations  can  acquire 
modified  commercial  and  nondevelopmenM  items  at  a  lower  cost  and  with 
faster  deliveries  when  buying  organizations  develop  acquisition  strategies  to 
effectively  manage  program  uncertainties.  In  commenting  on  acquisition 
strategies,  the  37  buying  organizations  identified  lessons  learned  that  generally 
could  also  be  applied  to  the  management  of  all  DoD  acquisitions,  that  is,  new 
development  efforts  as  well  as  modified  commercial  and  nondevelopmental  item 
acquisitions.  For  example,  the  buying  organizations  identified  91  lessons 
learned  that  related  to  developing  the  acquisition  strategy.  Of  the  91  lessons 
learned,  61  lessons  could  also  be  applied  to  the  management  of  acquisitions  in 
general,  such  as  the  following  lessons: 

•  program  offices  must  involve  all  interested  parties  early  in  the 
acquisition  process, 

•  contracting  officers  should  develop  achievable  schedules  that  include 
appropriate  funding  reserves  for  unforeseen  delays,  and 

•  contracting  officers  should  develop  realistic,  not  optimistic, 
schedules  to  satisfy  user  requirements. 
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Lessons  Learned  From  Acquiring  Modified  Commercial  and  Nondevelopmental 
Items  _ _ _ 


We  summarized  and  categorized  the  lessons  learned  by  critical  program 
management  element,  that  is,  program  definition,  program  structure,  and 
program  design  as  defined  in  DoD  Regulation  50(X).2-R.  Although  not 
specifically  described  in  DoD  Regulation  5000.2-R,  we  also  categorized  lessons 
learned  about  contracting  issues  because  of  the  significant  differences  that  may 
exist  in  contracting  in  a  commercial  environment  and  in  a  Government 
environment. 

Appendix  D  is  a  detailed  listing  of  many  of  the  lessons  learned  by  program 
management  element  for  the  19  modified  commercial  item  acquisitions 
reviewed,  and  Appendix  B  is  a  detailed  listing  of  the  lessons  learned  for  the 
23  nondevelopmental  item  acquisitions  reviewed.  Of  the  37  programs 
reviewed,  5  programs  involved  the  acquisition  of  both  modified  commercial  and 
nondevelopmental  items.  As  shown  in  Appendixes  D  and  E,  the  buying 
organizations  identified  many  lessons  learned  that  were  common  for  acquiring 
modified  commercial  and  nondevelopmental  items.  The  common  lessons 
occurred  because  organizations,  in  developing  acquisition  strategies,  were 
required  by  DoD  Regulation  5000.2-R  and  the  FAR  to  take  actions  necessary  to 
protect  the  Government’s  interests  and  to  resolve  program  uncertainties  related 
to  acquiring  modified  commercial  and  nondevelopmental  items  to  satisfy  user 
operational  requirements.  In  cases  where  the  lessons  learned  implemented  FAR 
and  Defense  Federal  Acquisition  Regulation  Supplement  policies,  we  referenced 
the  corresponding  policies  in  Appendixes  D  and  E.  In  all,  the  buying 
organizations  identified  241  lessons  learned  associated  with  their  efforts  to 
cost-efficiently  and  effectively  acquire  modified  commercial  and 
nondevelopmental  items. 

Program  Definition.  Program  definition  is  the  process  of  translating  broadly 
stated  mission  needs  into  a  set  of  operational  requirements  from  which  specific 
performance  specifications  are  derived.  DoD  Regulation  5000.2-R  says  that 
“use  of  that  process  helps  ensure  that  programs  are  well-defined  and  carefully 
structured  to  represent  a  judicious  balance  of  cost,  schedule,  and  performance; 
available  technology;  and  affordability  constraints.” 

Buying  organizations  for  4  of  the  37  programs  reviewed  identified  lessons 
learned  associated  with  program  definition.  One  of  the  lessons  learned  was  that 
users  need  to  participate  in  the  acquisition  process  as  early  as  possible  so  that 
they  understand  the  nature,  potential  limitations,  and  advantages  of  available 
modified  commercial  and  nondevelopmental  items.  That  is,  users  must  define 
system  requirements  that  allow  for  consideration  of  modified  commercial  and 
nondevelopmental  items  in  the  acquisition  process  as  is  required  by 
FAR  11.002(a)(2)(ii).  Further,  the  buying  organizations  stated  that  requirement 
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Lessons  Learned  From  Acquiring  ModiHed  Commercial  and  Nondevelopmental 

Items 


definitions  should  be  broad  enough  in  the  requests  for  proposal  to  increase 
competition  rather  than  limit  the  acquisitions  to  single-source  suppliers  as  is 
required  by  FAR  ll.(X)2(a)(l).  Consistent  with  FAR  10.002,  buying 
organizations  need  to  communicate  and  cooperate  with  users  early  to  ensure 
flexibility  in  system  requirements  while  providing  users  with  knowledge  of 
potential  modified  commercial  and  nondevelopmental  items  that  may  meet  their 
needs. 

Likewise,  as  requirements  are  refined  and  changed,  the  user  and  buying 
organization  must  recognize  the  limits  of  modified  commercial  and 
nondevelopmental  item  acquisitions  and  either  adjust  the  requirements  or  adjust 
the  acquisition  strategies.  For  example.  Office  of  the  Inspector  General,  DoD, 
Report  No.  95-193,  "DoD  Hotline  Allegations  Regarding  the  Naval  Special 
Warfare  Patrol  Coastal  Ship  and  Rigid  Inflatable  Boat  Acquisition  Programs," 
May  15,  1995,  reports  that  the  Navy  had  modified  requirements  for  the  Naval 
Special  Warfare  Patrol  Coastal  Ship  acquisition  program  beyond  the  point  where 
a  nondevelopmental  item  could  satisfy  the  requirements.  In  that  case,  the  Navy 
deviated  from  a  nondevelopmental  items  acquisition  approach  without 
conducting  necessary  risk  analyses  to  support  the  decision. 

Program  Structure.  Program  structure  identifies  management  elements  that 
are  necessary  to  structure  a  successful  program.  The  elements  address  what  the 
program  will  achieve  (program  goals);  how  the  program  will  be  developed, 
procured,  or  both  (acquisition  strategy);  how  the  program  will  be  evaluated 
against  what  was  intended  (test  and  evaluation);  and  what  resources  will  be 
needed  for  the  program  (life-cycle  resource  estimates).  Properly  tailored 
program  strategies  form  the  basis  for  sound  management  and  include  innovative 
ways  to  achieve  program  success. 

Buying  organizations  for  18  of  the  37  programs  reviewed  identified  lessons 
learned  associated  with  program  structure.  The  organizations  indicated  that 
failure  to  understand  the  marketplace  environment  can  affect  program  structures 
and  lead  to  flawed  assumptions  about  product  performance,  quality,  and 
logistical  support.  For  modified  commercial  items,  the  buying  organizations 
stated  that  commercial  suppliers  often  exaggerate  product  performance  and 
operating  parameters  for  competitive  reasons.  FAR  12.205  cautions  that 
“agencies  should,  as  part  of  market  research,  review  existing  product  literature 
generally  available  in  the  industry  to  determine  its  adequacy  for  purposes  of 
evaluation.”  Further,  commercial  suppliers  may  make  material  substitutions 
and  manufacturing  process  changes  during  production  that  may  affect  product 
quality  to  sustain  and  increase  their  share  of  the  product  marketplace.  Also, 
depending  on  market  sensitivities,  commercial  suppliers  sometimes  make 
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products  obsolete,  which  limits  the  extent  of  logistical  support  available  to 
users.  In  reference  to  nondevelopmental  items,  for  example.  Army  Audit 
Agency  Report  No.  NE-90-209,  "Acquisition  of  Nondevelopment^  Items  Fort 
Monmouth,  New  Jersey,"  September  11,  1990,  reports  that  the  Army  did  not 
properly  test  nondevelopmental  items  and  did  not  properly  support  the  logistics 
support  concepts  for  nondevelopmental  items. 

Accordingly,  the  buying  organizations  identified  lessons  learned  that  related  to 
acquisition  strategy  and  test  and  evaluation  that  can  be  used  to  avoid  and  defend 
against  those  markeQilace  realities. 

Acquisition  Strategy.  Buying  organizations  stressed  the  importance  of 
devising  acquisition  strategies  to  manage  program  uncertainties  identified  in  risk 
management  plans  that  may  impact  product  performance,  quality,  and  logistical 
support. 


Performance.  In  the  contract  request-for-proposal  process,  the 
buying  organizations  indicated  that  commercial  suppliers  often  exaggerate 
product  performance  and  operating  capabilities  to  obtain  competitive 
advantages.  FAR  12.205  cautions  buying  organizations  to  review  product 
literature  to  determine  its  adequacy  for  purposes  of  evaluation  through  market 
research  to  avoid  such  problems.  The  buying  organizations  also  described  other 
management  actions  taken  in  the  acquisition  strategies  to  verify  and  validate 
product  performance  and  operating  capabilities  that  were  linked  to  requirements 
development,  risk  management,  product  testing,  commercial  and 
nondevelopmental  product  integration,  and  contractor  submissions.  Examples 
of  lessons  learned  follow: 

•  Buying  organizations  must  adequately  test  commercial  products  to 
ensure  that  the  products  live  up  to  their  specifications  because  commercial 
specifications  tend  to  be  optimistic. 

•  Contracting  officers  should  require  verification  testing  of  contractor- 
furnished  product  performance  data. 

•  Buying  organizations  should  fully  understand  commercial  product 
procurement  and  testing  information. 
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Quality.  To  maintain  product  price  competitiveness,  the  buying 
organizations  indicated  that  commercial  suppliers  often  made  material 
substitutions  and  manufacturing  process  changes.  As  a  result,  product  quality 
could  change  during  the  item  life-cycle.  Recognizing  those  possibilities,  the 
buying  organizations  described  management  actions  taken  in  their  acquisition 
strategies  to  maintain  product  integrity  for  modified  commercial  and 
nondevelopmental  items  through  test  strategies,  contracting  documents,  and 
logistical  support  plans.  Examples  of  lessons  learned  follow: 

•  Buying  organizations  should  recognize  that  parts  obtained  from 
different  commercial  suppliers  may  not  function  the  same  as  original  equipment 
manufacturers’  parts. 

•  Buying  organizations  should  understand  that  configuration  tracking 
and  material  support  can  become  difficult  when  commercial  suppliers  make 
changes  to  equipment  unilaterally. 

•  Contracting  officers  should  consider  including  contract  provisions 
requiring  commercial  suppliers  to  update  technical  documentation  to  reflect 
configuration  management  changes. 

Logistical  Support.  Buying  organizations  reported  that 
continued  logistical  support  for  items  after  fielding  is  a  significant  area  of 
uncertainty  for  modified  commercial  and  nondevelopmental  item  acquisitions. 
While  limited  logistical  support  after  item  deliveries  may  be  sufficient  for 
commercial  consumers,  the  buying  organizations  indicated  that  most  often  the 
limited  logistical  support  is  not  sufficient  for  DoD  life-cycle  logistical  support 
requirements.  Specifically,  commercial  suppliers  often  make  hardware  and 
software  improvements  and,  as  previously  indicated  in  this  report,  substitute 
components  or  change  production  processes  to  sustain  and  increase  their 
marketshare.  Depending  on  marketplace  sensitivities,  the  buying  organizations 
indicated  that  logistical  support  after  item  deliveries  for  modified  commercial 
and  nondevelopmental  items  can  vary  from  2  years  to  5  years.  Accordingly,  the 
buying  organizations  identified  efforts  to  manage  program  uncertainties  in  their 
acquisition  strategies  to  enable  users  to  operate  and  maintain  acquired  systems 
over  the  required  items’  life-cycles  when  contract  warranties  expire  and  when 
maintenance  and  supply  support  cease.  Buying  organizations  need  to  implement 
efforts  to  manage  program  uncertainties  in  their  acquisition  strategies  because 
commercial  suppliers  are  reluctant  to  disclose  proprietary  design  and  production 
information.  Recognizing  the  logistical  support  problem,  buying  organizations 
described  actions  taken  in  their  lessons  learned  that  would  mitigate  logistical 
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Lessons  Learned  From  Acquiring  ModiHed  Commercial  and  Nondevelopmental 
Items  _ _ _ _ _ 


uncertainties  through  maintenance  support,  warranties  and  data  rights, 
configuration  management,  documentation,  and  parts  provisioning.  Examples 
of  lessons  learned  follow: 

•  Contracting  officers  should  consider  including  contract  option(s)  for 
parts  and  technical  data  buyouts  to  support  future  user  logistics  requirements. 

•  Buying  organizations  should  consider  requiring  that  a  third  party  hold 
technical  data  in  escrow  for  Government  use  after  commercial  supplier  support 
ends  if  the  product  life-cycles  extend  beyond  commercial  supplier  support. 

•  Buying  organizations  need  to  verify  that  commercial  supplier  parts 
support  is  longer  than  the  anticipated  part  life-cycle. 

Test  and  Evaluation.  Buying  organizations  emphasized  the  need, 
consistent  with  commercial  practices,  to  include  sufficient  testing  before  and 
after  contract  award  to  ensure  that  modified  commercial  and  nondevelopmental 
items  will  work  in  the  military  environment,  which  can  be  significantly  different 
from  the  environment  for  which  commercial  suppliers  developed  the  items. 

Prior  audit  reports  identify  that  need  as  a  significant  acquisition  management 
problem.  The  buying  organizations  also  indicated  that  program  offices  should 
pay  particular  attention  to  how  commercial  hardware  and  software  specification 
products  will  be  integrated  into  military  specification  platforms  and  should 
identify  the  resources  and  testing  that  buying  organizations  will  need  to 
accomplish  the  integration.  Examples  of  lessons  learned  follow: 

•  Buying  organizations  must  recognize  that  early  test  and  evaluation  of 
commercial  components  is  important  when  commercial  suppliers  are  modifying 
a  commercial  system. 

•  Buying  organizations  should  use  conformance-testing  of  commercial 
products  in  the  source-selection  process  to  provide  assurance  that  products 
function  properly  in  a  specific  integrated  system  configuration  (the  larger  and 
more  complex  the  system,  the  higher  the  risk). 

•  Buying  organizations  should  consider  requiring  the  performance  of 
operational  testing  and  evaluation  as  part  of  the  source-selection  process. 

Program  Design.  Program  design  establishes  the  basis  for  a  comprehensive, 
structured,  integrated,  and  disciplined  approach  to  the  life-cycle  design  of  major 
weapons  and  automated  information  systems.  Through  the  system  engineering 
process,  the  program  manager  is  to  translate  operational  needs,  requirements,  or 
both  into  a  system  solution  that  includes  the  design,  manufacturing,  test  and 
evaluation,  and  support  processes  and  products. 
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Lessons  Learned  From  Acquiring  ModiHed  Commercial  and  Nondevelopmental 

Items 


Buying  organizations  for  14  of  the  37  programs  reviewed  identified  lessons 
learned  associated  with  program  design.  Buying  organizations  indicated  that 
program  offices  should  implement  acquisition  strategies  that  recognize  the 
ability  to  upgrade  or  second  source  modified  commercial  and  nondevelopmental 
items  as  the  program  evolves.  Specific  lessons  learned  included  the  following: 

•  Buying  organizations  need  to  acquire  systems  that  use  open  systems 
architecture  to  negate  the  need  for  commercial  product  data  to  avoid  using 
single-source  products  or  product  features  that  are  unique  to  a  single 
manufacturer. 

•  Buying  organizations  should  purchase  the  necessary  technical  data 
rights  for  electrical  interfaces,  protocols,  and  performance  parameters  of  critical 
circuit  card  subcomponents  when  obtaining  open  system  architecture 
components  to  preserve  the  opportunity  to  accomplish  later  growth  options  and 
engineering  change  proposals  in  an  inexpensive  manner. 

The  buying  organizations  also  stated  that  another  tool  to  reduce  program 
uncertainties  during  program  design  is  prototyping.  Through  prototyping, 
buying  organizations  can  identify  product  interoperability  problems  associated 
with  using  other  commercial  suppliers  to  supply  the  same  or  similar  item. 

Contracting.  Contracting  describes  work  requirements,  specifications,  costs, 
deliverables,  administration,  restrictions,  and  limitations  between  the 
Government  and  contractors.  The  selection  of  contractual  sources  and  contract 
requirements  must  be  well  thought  out  and  tailored  to  accomplish  stated 
objectives  while  ensuring  an  equitable  sharing  of  program  management  risks. 

Buying  organizations  for  12  of  the  37  programs  reviewed  identified  lessons 
learned  associated  with  contracting.  The  buying  organizations  pointed  out  the 
need  for  personnel  associated  with  the  selection,  award,  and  administration  of 
contracts  for  modified  commercial  and  nondevelopmental  items  to  understand 
the  limitations  and  advantages  of  acquiring  those  items  and  to  provide  oversight 
of  the  program  accordingly.  Specific  lessons  learned  were  as  follows: 

•  Contracting  officers  should  consider  the  use  of  operational 
demonstrations  during  source  selection. 

•  Contracting  officers  should  compare  the  quality  of  the  product  and 
future  service  requirements  against  the  contractors'  past  performance  to  meet 
current  needs. 

•  Contracting  officers  should  clearly  define  the  contract  administration 
organization’s  role  in  product  quality  assurance. 
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Lessons  Learned  From  Acquiiing  ModiHed  Commercial  and  Nondevelopmental 
Items  _ _ _ _ 


Managing  Program  Uncertainties 

The  majority  of  lessons  learned  discussed  previously  and  in  Appendixes  D  and 
E  are  actions  to  manage  program  uncertainties.  Based  on  discussions  with 
buying  organization  officials  and  reviews  of  supporting  documentation,  the 
biggest  challenge  facing  program  mangers  acquiring  modified  commercial  and 
nondevelopmental  items  is  understanding  the  program  uncertainties  involved 
with  those  types  of  acquisitions  and  effectively  managing  them.  The  buying 
organizations  designed  strategies  to  either  avoid,  reduce,  transfer,  or  assume 
program  acquisition  uncertainties  relating  to  program  definition,  program 
structure,  program  design,  and  contracting.  Buying  organization  actions  taken 
to  manage  acquisition  uncertainties  did  not  attempt  to  eliminate  the  uncertainties 
but  rather  attempted  to  monitor  and  manage  uncertainties  to  minimize  their 
effects  on  programs.  Examples  of  management  actions  that  buying 
organizations  can  take  on  future  acquisitions  of  modified  commercial  and 
nondevelopmental  items  include: 

•  placing  more  reliance  on  use  of  proven  contractor  logistical  support 
networks; 

•  developing  requirements  that  take  maximum  advantage  of 
commercial  items  with  previous  commercial  testing;  and 

•  making  the  prime  integrating  contractor  a  major  stakeholder  in  the 
acquisition. 


Progress  Made 


When  compared  with  shortcomings  identified  in  prior  audit  reports,  buying 
organizations  have  made  progress  in  strengthening  acquisition  strategies  used  to 
acquire  modified  commercial  and  nondevelopmental  items.  After  greater 
exposure  to  those  types  of  acquisitions,  buying  organizations  realize  that 
commercial  suppliers  will  not  always  be  forthcoming  about  product 
performance,  reliability,  and  subsequent  logistical  support.  As  a  result,  buying 
organizations  are  now  addressing  in  their  acquisition  strategies  the  need  for 
commercial  suppliers  to  substantiate  with  test  results  their  claims  for  product 
performance,  quality,  and  logistical  support.  In  addition,  buying  organizations 
are  more  attuned  to  users’  needs  and  requirements  by  making  efforts  to  include 
them  on  integrated  product  teams  early  in  the  acquisition  process.  Such  early 
involvement  is  essential  in  the  effective  use  of  commercial  items.  Because  of 
past  integration,  interface,  and  interoperability  problems  in  which  modified 
commercial  and  nondevelopmental  items  become  part  of  larger  systems,  buying 
organizations  are  also  placing  greater  emphasis  on  actions  that  management  can 
take  to  mitigate  against  the  uncertainties  associated  with  those  problems. 
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Lessons  Learned  From  Acquiring  ModiHed  Commercial  and  Nondevelopmental 

Items 


Conclusion 


Buying  modified  commercial  and  nondevelopmental  items  offers  DoD  buying 
organizations  the  opportunity  to  acquire  items  at  a  lower  cost  and  with  faster 
deliveries.  However,  to  obtain  those  benefits,  buying  organizations  should 
recognize  the  program  uncertainties  involved  with  the  acquisitions  that  affect 
product  performance,  quality,  and  logistical  support. 

Based  on  our  audit  results  and  analysis  of  prior  audit  reports,  effective  use  of 
modified  commercial  and  nondevelopmental  items  requires  that  buying 
organizations  develop  and  implement  acquisition  strategies  that: 

•  ensure  that  users  of  the  items  are  involved  early  in  the  acquisition 
process  and  understand  the  nature,  potential  limitations,  and  advantages  of 
modified  commercial  and  nondevelopmental  items.  Only  in  that  way  can  the 
user  and  acquisition  community  maintain  the  flexibility  that  is  needed  in 
establishing  requirements  to  make  effective  use  of  the  modified  commercial  and 
nondevelopmental  item. 

•  include  sufficient  testing,  consistent  with  commercial  practices, 
before  and  after  contract  award  to  ensure  that  the  modified  commercial  and 
nondevelopmental  items  will  work  in  the  intended  military  environment. 

•  focus  attention  on  hardware  and  software  integration,  interface,  and 
interoperability  testing  efforts  and  on  the  resources  that  are  needed  to 
successfully  conduct  the  tests  when  modified  commercial  and  nondevelopmental 
items  are  integral  components  and  subsystems  of  larger  systems. 

•  address  how  modified  commercial  and  nondevelopmental  items  will 
be  maintained  and  supported  when  fielded  because  Government  access  to 
technical  data  will  most  likely  be  limited,  and  availability  of  parts  supplied  may 
cease  before  the  intended  military  life-cycles  of  the  items  expire. 

•  realistically  recognize  the  extent  of  time  and  money  necessary  to 
integrate  modified  commercial  and  nondevelopmental  items  into  military  systems 
and  applications. 
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In  addition,  buying  organizations  must  not  assume  that  because  an  item  is  a 
modified  commercial  or  nondevelopmental  item  it  does  not  require  Government 
oversight  and  management.  Buying  organizations  must  tailor  oversight  and 
management  to  the  circumstances  of  the  particular  acquisition  and  must  base 
them  on  an  assessment  of  the  program  uncertainties  affecting  the  acquisition. 
Specifically,  buying  organizations  need  to: 

•  identify  program  uncertainties  associated  with  a  particular  modified 
commercial  or  nondevelopmental  acquisition  and  make  plans  to  manage  the 
program  uncertainties, 

•  ensure  that  all  buying  organization  personnel  associated  with  the 
selection  and  award  of  a  modified  commercial  and  nondevelopmental  item 
understand  the  limitations  and  advantages  of  that  type  of  acquisition  and  provide 
the  oversight  of  the  program  accordingly,  and 

•  ensure  that  all  personnel  involved  with  acquisitions  of  modified 
commercial  and  nondevelopmental  items  understand  the  need  to  be  able  to 
upgrade  or  second  source  Aose  items  as  the  program  evolves  over  time. 

DoD’s  Acquisition  Reform  Office  plans  to  disseminate  lessons  learned  from 
commercial  and  nondevelopmental  item  acquisitions  through  the  internet 
Commercial  Advocates  Form  homepage  at  http://www.acq.osd.mil/ar/cadv.htm. 
Through  electronic  postings,  DoD  plans  to  provide  buying  organizations  with 
useful  information  on  how  to  acquire  modified  commercial  and 
nondevelopmental  items  from  commercial  suppliers. 
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Appendix  A.  Audit  Process 


Scope 


We  conducted  this  program  results  audit  from  October  1996  through  June  1997 
in  accordance  with  auditing  standards  issued  by  the  Comptroller  General  of  the 
United  States,  as  implemented  by  the  Inspector  General,  DoD.  We  reviewed 
10  Army,  23  Navy,  and  4  Air  Force  acquisition  programs  in  which  the  Military 
Department  was  acquiring  modified  commercial  and  nondevelopmental  items  as 
entire  systems,  subsystems,  or  major  components.  The  37  acquisition  programs 
consisted  of  6  Acquisition  Category  I  programs,  4  Acquisition  Category  n 
programs,  16  Acquisition  Category  m  programs,  and  11  non-acquisition 
category  programs.  Appendix  F  defines  the  three  acquisition  categories.  We 
estimated  that  total  funds  spent  for  the  37  programs  will  exceed  $16  billion.  At 
the  program  offices  for  modified  commercial  and  nondevelopmental  items,  we 
identified  lessons  learned  in  developing  acquisition  strategies  for  program 
definition;  program  structure;  program  design;  contracting;  program 
assessments;  and  decisions,  reviews,  and  periodic  reporting.  Accordingly,  we 
included  tests  of  management  controls  as  considered  necessary. 


Methodology 


We  used  available  information  from  ongoing  and  past  management  efforts  of 
modified  commercial  and  nondevelopmental  item  acquisitions  to  identify  lessons 
learned.  In  addition,  we  visited  22  of  the  37  buying  organizations  to  discuss 
specific  lessons  learned.  We  also  reviewed  audit  reports  addressing  acquisitions 
of  modified  commercial  and  nondevelopmental  items  to  determine  whether  the 
acquisition  community  is  making  progress  in  developing  acquisition  strategies 
that  avoid  some  of  the  acquisition  difficulties  identified  in  earlier  audit  reports. 
We  did  not  rely  on  computer-processed  data  or  statistical  sampling  procedures 
to  develop  conclusions  on  this  audit. 

Contacts  During  the  Audit.  We  visited  or  contacted  individuals  and 
organizations  within  the  DoD.  Further  details  are  available  on  request. 


16 


Appendix  A»  Audit  Process 


Management  Control  Program 


Requirement  for  Management  Control  Review.  DoD  Directive  5010.38, 
"Management  Control  (MC)  Program,"  August  26,  1996,  requires  DoD 
managers  to  implement  a  comprehensive  system  of  management  controls  that 
provides  reasonable  assurance  that  programs  are  operating  as  intended  and  to 
evaluate  the  adequacy  of  the  controls. 

Scope  of  Review  of  the  Management  Control  Program.  We  limited  our 
review  because  of  the  relevant  coverage  in  Inspector  General,  DoD,  Report 
No.  96-028,  "Implementation  of  the  DoD  Management  Control  Program  for 
Major  Defense  Acquisition  Programs,"  November  28,  1995.  The  report 
discusses  the  effectiveness  of  the  management  control  program  that  the  Defense 
Acquisition  Executive  and  Component  Acquisition  Executives  used  for  major 
Defense  acquisition  programs.  The  report  concludes  that  the  acquisition 
community  had  not  effectively  integrated  DoD  Management  Control  Program 
requirements  into  its  management  assessment  and  reporting  processes.  As  a 
result  of  the  report  recommendations,  the  Under  Secretary  of  Defense  for 
Acquisition  and  Technology  integrated  DoD  Directive  5010.38  requirements 
into  the  March  15,  1996,  revision  to  DoD  Directive  5000.1,  "Defense 
Acquisition,"  and  DoD  Regulation  5000.2-R.  Acquisition  managers  are  now  to 
use  program  cost,  schedule,  and  performance  parameters  as  control  objectives  to 
implement  the  DoD  Directive  5010.38  requirements.  The  managers  are  to 
identify  material  weaknesses  through  deviations  from  approved  acquisition 
program  baselines  and  exit  criteria  in  the  "Defense  Acquisition  Executive 
Summary"  report.  Consequently,  we  limited  our  review  to  management 
controls  directly  related  to  the  critical  program  management  elements  of  the 
acquisition  programs  reviewed. 

Adequacy  of  Management  Controls.  Management  controls  were  adequate  in 
that  we  did  not  identify  any  systemic  management  control  weakness  applicable 
to  our  primary  audit  objective. 
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During  the  last  8  years,  the  General  Accounting  Office;  the  Office  of  the 
Inspector  General,  DoD;  the  Army  Audit  Agency;  and  the  Air  Force  Inspector 
General  have  issued  16  reports  that  cover  issues  related  to  this  audit.  The  audits 
addressed  the  process  associated  with  either  nondevelopmental  item  acquisitions 
or  issues  related  to  a  specific  nondevelopmental  item  acquisition. 


General  Accounting  Office 

Report  No.  GAO/NSIAD-97-3  (OSD  [Office  of  the  Secretary  of  Defense]  Case 
No.  1215),  “The  Government’s  Market  Research  Efforts,”  October  11,  1996, 
reports  that  Government  market  research  efforts  varied  widely,  but  that  the 
variances  appeared  to  be  appropriate.  For  instance,  the  Department  of  Defense 
emphasized  the  use  of  market  research  for  commercial  items  and 
nondevelopmental  items  more  than  the  civilian  agencies  did  before  the  Federal 
Acquisition  Streamlining  Act  of  1994,  primarily  because  of  statutory 
requirements  that  applied  to  the  Department  of  Defense.  Results  from  21 
contracts  reviewed  showed  that  agencies: 

•  obtained  commercial  or  nondevelopmental  items  in  16  of  the  21 
contracts,  although  for  3  of  16  contracts,  market  research  was,  for  legitimate 
reasons,  not  performed  and 

•  used  market  research  to  obtain  Government-unique  items  or  services 
in  5  of  the  21  contracts. 

Report  No.  GAO/NSIAD-96-4  (OSD  Case  No.  1033),  "Medium  Trucks  Passed 
Key  Operational  and  Technical  Tests,"  January  8,  1996,  reports  that  overall,  the 
Army  Family  of  Medium  Tactical  Vehicle  trucks  performed  satisfactorily; 
however,  many  of  the  test  vehicles  were  not  produced  on  the  production  line  or 
were  retrofitted  to  correct  past  deficiencies.  The  report  states  that  "if  the 
Army's  comparison  tests  include  full-production  and  retrofitted  trucks,  [the 
Army]  should  have  adequate  assurance  that  the  trucks  continue  to  meet  the 
Army's  performance  and  reliability,  availability,  and  maintainability 
requirements. "  The  Department  of  Defense  concurred  with  the  report.  The 
Family  of  Medium  Tactical  Vehicle  trucks  program  is  an  Army 
nondevelopmental  item  acquisition. 
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Report  No.  GAO/NSIAD-96-2  (OSD  Case  No.  1030),  "Hunter  System  Is  Not 
Appropriate  For  Navy  Fleet  Use,"  December  1,  1995,  concludes  that  the  Joint 
Tactical  Unmanned  Aerial  Vehicles  project  office  was  proceeding  with  the 
acquisition  of  the  Hunter  shipboard  variant  unmanned  aerial  vehicle  even  though 
Navy  fleet  commanders  stated  that  they  did  not  want  that  system  on  naval  ships. 
The  report  recommended  a  halt  to  the  acquisition  of  Hunter  shipboard  variants 
unmanned  aerial  vehicle  system  until  the  Navy  completes  an  assessment  of 
shipboard  unmanned  aerial  vehicle  requirements,  determines  whether  the  Hunter 
system  will  meet  the  requirements,  and  determines  whether  fleet  commanders 
will  use  the  Hunter  system.  The  Department  of  Defense  comments  were 
responsive  to  the  report  recommendations.  The  Hunter  shipboard  variant 
unmanned  aerial  vehicle  system  is  a  nondevelopmental  item  acquisition. 

Report  No.  GAO/NSIAD-95-161  (OSD  Case  No.  3349),  "Maneuver  System 
Schedule  Includes  Unnecessary  Risk,"  September  15,  1995,  reports  that  the 
Joint  Tactical  Unmanned  Aerial  Vehicles  project  office  planned  to  begin 
production  of  the  Maneuver  System  without  adequate  assurance  that  the  system 
could  meet  operational  requirements.  Recommendations  for  the  Pioneer  and 
Hunter  nondevelopmental  item  acquisitions  included  changing  the  Maneuver 
System's  acquisition  strategy  to  require  that  sufficient  operational  tests  be 
conducted  before  the  start  of  low-rate  initial  production.  The  Department  of 
Defense  did  not  commit  to  performing  the  operational  tests  before  low-rate 
initial  production. 

Report  No.  GAO/NSIAD-89-51  (OSD  Case  No.  7923),  "DoD  Efforts  Relating 
to  Nondevelopmental  Items,"  February  7,  1989,  concludes  that  the  Department 
of  Defense  could  strengthen  its  efforts  to  buy  nondevelopmental  items.  Report 
recommendations  include  making  resources  available  to  ensure  proper  training 
for  acquisition  personnel  concerning  nondevelopmental  items  and  determining 
whether  a  regulatory  change  is  needed  relating  to  exemptions  from  cost  or 
pricing  data  requirements  based  on  established  catalog  or  market  prices.  The 
General  Accounting  Office  did  not  request  or  obtain  comments  from  the 
Department  of  Defense. 
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Inspector  General,  DoD 


Inspector  General  Report  No.  97-145,  "Purchasing  Commercial  Products," 

May  23,  1997,  reports  that  the  DoD  buying  commands  were  purchasing 
commercial  products  when  practicable.  The  Army,  Navy,  Air  Force,  and 
Defense  Logistics  Agency  buying  commands  conducted  market  research  to 
identify  commercially  available  products  to  satisfy  their  needs.  The  report  had 
no  recommendations. 

Inspector  General  Report  No.  96-111,  "Allegations  Involving  the  Procurement 
of  the  Hunter  Unmanned  Aerial  Vehicle-Short-Range  System,"  May  7,  1996, 
reports  that  the  Hunter  Unmanned  Aerial  Vehicle-Short-Range  low-rate  initial 
production  system  did  not  conform  to  contract  requirements,  did  not  have 
adequate  reliability,  and  did  not  experience  system  operational  testing.  The 
report  did  not  make  recommendations  for  corrective  action  because  of  the 
scheduled  termination  of  the  program.  The  Hunter  unmanned  aerial  vehicle 
program  is  a  nondevelopmental  item  acquisition. 

Inspector  General  Report  No.  96-020,  "Contractor's  Performance  on  the  Family 
of  Medium  Tactical  Vehicles  Program,"  November  1,  1995,  reports  that  the 
contractor  for  the  nondevelopmental  item  acquisition  of  the  Family  of  Medium 
Tactical  Vehicles  had  not  attained  the  level  of  production  necessary  to  meet 
contractual  delivery  schedules.  Recommendations  include  revising  the  delivery 
schedule,  holding  the  contractor  accountable  to  meet  a  revised  delivery 
schedule,  including  a  liquidation  damages  clause  in  future  Family  of  Medium 
Tactical  Vehicles  contracts,  and  determining  the  need  for  a  second  source  for 
the  Family  of  Medium  Tactical  Vehicles.  The  Army  comments  were  responsive 
to  the  intent  of  the  report  recommendations. 

Inspector  General  Report  No.  96-005,  "Quick-Reaction  Audit  Report  on 
Conditional  Acceptance  of  Medium  Tactical  Vehicles,"  October  12,  1995, 
reports  that  the  Family  of  Medium  Tactical  Vehicles  program  office  had 
conditionally  accepted  incomplete  nondevelopmental  vehicles  from  the 
contractor  and  planned  to  conditionally  accept  additional  vehicles,  which  was 
not  in  the  best  interest  of  the  Government.  The  report  recommended  that  the 
Army  stop  conditionally  accepting  vehicles  under  the  Family  of  Medium 
Tactical  Vehicles  program.  In  December  195,  the  Dqmty  Assistant  Secretary 
of  the  Army  (Procurement)  concurred,  stating  that  the  contractor  had  refined  his 
production  processes  and  would  retrofit  the  552  conditionally  accepted  vehicles. 
However,  as  of  August  31,  1997,  the  Army  had  conditionally  accepted  another 
4,960  vehicles.  The  Army  has  fully  accepted  only  57  vehicles. 
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Inspector  General  Report  No.  95-193,  "DoD  Hotline  Allegations  Regarding  the 
Naval  Special  Warfare  Patrol  Coastal  Ship  and  Rigid  Inflatable  Boat  Acquisition 
Programs,"  May  15,  1995,  reports  that  the  Naval  Sea  Systems  Command 
deviated  from  a  nondevelopmental  items  acquisition  approach  without 
conducting  necessary  risk  analyses  to  support  the  decision.  The  Navy  had 
modified  the  ship  requirements  beyond  the  point  at  which  a  nondevelopmental 
item  could  satisfy  them.  The  report  recommendation  directed  the  Navy  to 
properly  document  as  lessons  learned  the  events  that  occurred  during  the 
acquisition  of  the  Patrol  Coastal  Ship  and  Rigid  Inflatable  Boat  Acquisition 
Programs  and  to  disseminate  the  lessons-leamed  to  appropriate  Navy  acquisition 
officials.  The  Naval  Sea  Systems  Command  prepared  a  lessons  learned 
document  as  recommended.  The  lessons  learned  were: 

•  In  general,  combining  requirements  leads  to  larger  platforms.  A 
detailed  requirements  review  with  the  program  sponsor  and  operators  could 
have  eliminated  the  perception  that  the  platform  was  too  large  or  was  larger  than 
ne^ed  to  meet  the  requirements. 

•  When  the  decision  to  commission  the  Patrol  Coastal  Ship  and  Rigid 
Inflatable  Boat  Acquisition  Programs  was  made,  the  program  office  should  have 
conducted  an  indq)endent  impact  study  and  taken  the  time  necessary  to  address 
the  concerns  of  Commander,  Naval  Sea  Systems  Command. 

•  The  Naval  Sea  Systems  Command  should  not  have  accelerated  the 
Rigid  Inflatable  Boat  program  design,  but  should  have  taken  the  time  necessary 
to  minimize  the  risk  in  die  design. 

•  Because  the  requirements  within  the  Operational  Reqi^ements 
Document  were  more  restrictive  than  those  within  the  Non- Acquisition  Program 
Definition  Document,  the  Naval  Special  Warfare  Rigid  Inflatable  Boat  program 
office  should  have  determined  areas  in  which  requirements  could  have  been 
compromised  without  loss  of  mission  effectiveness. 

•  The  Naval  Sea  Systems  Command  did  not  perform  an  adequate  risk 
analysis,  which  would  have  shown  the  high  technical  risk  driven  by  the  failure 
of  commercial  vessels  to  meet  the  Operational  Requirements  Document  and  the 
compressed  schedule. 

•  A  low-rate  initial  production  Rigid  Inflatable  Boat  contract  should 
have  been  awarded  as  a  first  article  contract  with  only  the  first  2  boats 
authorized  to  be  built  and  tested  before  the  remaining  16  boats  in  the  contract 
could  be  built.  Limiting  the  initial  production  contract  to  the  first  two  boats  for 
test  purposes  would  have  minimized  the  production  risk  for  the  remaining  boats. 

•  The  Naval  Special  Warfare  Rigid  Inflatable  Boat  program  office  and 
the  Naval  Sea  Systems  Command  should  have  developed  a  realistic  obligation 
program  plan. 
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Inspector  General  Report  No.  92-107,  "Effectiveness  of  DoD  Use  of 
Nondevelopment  Items  in  Major  Defense  Acquisition  Programs,"  June  22,  1992, 
reports  that  program  offices  for  major  Defense  acquisition  programs  did  not 
consider  use  of  nondevelopmental  items  to  the  maximum  extent  practicable. 
Nondevelopmental  item  investigations  were  made  at  system  and  parts  levels,  but 
generally  were  not  made  at  the  subsystem  and  component  levels.  The  report 
recommended  that  the  Defense  Acquisition  Board  committees  address  in  the 
integrated  program  assessment  the  nondevelopmental  item  candidates  for 
research  and  results  of  the  nondevelopmental  items  researched  at  Milestones  I 
and  n,  respectively.  Management  implemented  the  report  recommendations. 


Army  Audit  Agency 


Army  Audit  Agency  Rq)ort  No.  NE-91-204,  "Acquisition  of  Nondevelopmental 
Items,"  June  17,  1991,  reports  that  Army  key  decisions,  such  as  streamlining 
the  acquisition  process  and  entering  production  without  a  research  and 
development  phase,  were  not  adequately  supported.  The  Army  did  not  comply 
with  policies  and  procedures  requiring  technical  and  user  testing  of 
nondevelopmental  items  before  production.  Recommendations  include: 

•  reemphasizing  the  need  for  program  managers  to  perform  complete 
market  investigations  in  support  of  acquisition  strategies  for  nondevelopmental 
items; 

•  clarifying  Army  policy  concerning  the  concept  formulation  process 
for  nondevelopmental  items; 

•  eliminating  integration  projects  as  a  category  of  nondevelopmental 
items;  and 

•  reemphasizing  to  milestone  decision  review  members  that  they  ensure 
that  the  acquisition  strategies  fully  explain  and  justify  all  streamlining  actions, 
that  the  results  of  independent  evaluations  of  market  investigations  are  available 
for  review,  and  that  the  test  plans  are  completed  before  decision  reviews 
authorizing  the  award  of  nondevelopmental  item  contracts. 

The  Army  concurred  with  the  report  recommendations.  The  report  summarizes 
the  audit  results  identified  in  the  next  three  Army  audit  reports. 
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Anny  Audit  Agency  Report  No.  NE-90-209,  "Acquisition  of  Nondevelopmental 
Items  Fort  Monmouth,  New  Jersey,"  September  11,  1990,  reports  that  the 
Army  did  not  properly  test  nondevelopmental  items  and  did  not  properly 
support  the  logistics  support  concepts  for  nondevelopmental  items.  Also, 
acquisition  strategies  did  not  adequately  address  steps  to  tailor  the  acquisition 
process.  Report  recommendations  include: 

•  establishing  controls  to  ensure  that  the  acquisition  strategy  addresses 
all  phases  of  the  acquisition  process  for  nondevelopmental  item  projects, 

•  establishing  controls  to  ensure  that  technical  and  operational  testing 
and  evaluation  were  done  before  nondevelopmental  items  enter  production 
unless  independent  reviews  conclude  that  tests  are  not  needed,  and 

•  analyzing  support  concepts  for  nondevelopmental  items  to  ensure 
proper  nondevelopmental  item  support. 

The  Army  concurred  with  the  recommendations  and  established  procedures  that 
made  Army  buying  organizations  address  aU  phases  of  the  acquisition  process, 
perform  technical  and  operational  testing  before  nondevelopmental  items  enter 
production,  and  analyze  support  concepts. 

Army  Audit  Agency  Report  No.  EC-90-211,  "Acquisition  of  Nondevelopmental 
Items  -  Program  Executive  Officer,  Combat  Support  and  U.S.  Army 
Tank- Automotive  Command,  Warren,  Michigan,"  August  21,  1990,  concludes 
that  most  nondevelopmental  items  fielded  or  about  to  be  fielded  were  not 
properly  evaluated  for  reliability.  The  report  recommended  that  all 
nondevelopmental  items  acquisitions  with  approved  requirements  documents 
fi-om  January  1982  through  September  1986  establish  a  plan  to  evaluate  whether 
cost-effective  reliability  data  can  be  obtained  for  nondevelopmental  items  rebuys 
to  determine  the  quantitative  reliability  performance  of  the  systems. 

Management  concurred  with  the  recommendation  and  made  reliability  studies 
for  nondevelopmental  item  rebuys. 

Army  Audit  Agency  Report  No.  NE-90-206,  "Acquisition  of  Nondevelopmental 
Items  U.S.  Army  Natick  Research,  Development  and  Engineering  Center, 
Natick,  Massachusetts,"  March  19,  1990,  reports  that  the  Army  often  based 
acquisition  strategies  on  incomplete  information  because  the  Army  did  not 
properly  conduct  market  investigations  and  concept  formulation  analyses.  The 
report  recommended  that  the  Army: 

•  identify  the  category  of  nondevelopmental  item  projects  in 
acquisition  strategies, 

•  conduct  and  document  market  investigations  for  all  acquisition 
projects. 
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•  establish  procedures  to  ensure  that  market  investigations  are  fully 
coordinated  with  independent  evaluators,  and 

•  ensure  that  proper  market  investigation  and  concept  formulation 
packages  have  been  completed  for  all  pre-Milestone  I  projects. 

The  Army  concurred  with  the  recommendations  and  implemented  procedures 
that  made  Army  buying  organizations  conduct  market  investigations  and  concept 
formulation  andyses  before  acquiring  nondevelopmental  items. 


Air  Force  Inspector  General 


Air  Force  Inspector  General  Report  No.  PN  95-506,  "Nondevelopmental  Items 
(NDI)  in  Air  Force  Acquisition,"  January  3,  1996,  reports  that  the  d^loyment 
of  methodologies  and  best  practices  to  overcome  the  challenges  traditionally 
associated  with  nondevelopmental  item  acquisitions  were  missing  or 
inconsistent.  Report  recommendations  include  expanding  training  and  tools  to 
program  managers  and  other  specialists  to  encourage  a  renewed  awareness  of 
nondevelopmental  item  solutions;  gathering  and  dissenunating 
nondevelopmental  item  best  practices  to  program  offices,  the  test  community, 
and  major  commands;  and  ensuring  representation  from  the  test  community  in 
new  start  acquisition  forums. 
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Appendix  C.  Modified  Commercial  and 
Nondevelopmental  Item  Acquisitions  Reviewed 


Modified 

Commercial 

Nondeveloomental 

Department  of  the  Army 

Biological  Integrated  Detection  System 

✓ 

Cargo  Utility  Commercial  Vehicle 

Y 

Cargo  Utility  Global  Positioning  System  Receiver 

V 

Communications-Electronics  Command 

Commercial  Communications  Technology  Lab 

✓ 

Deployable  Universal  Combat  Earthmover 

V' 

Lightweight  Multiband  Satellite  Terminal 

V' 

Lightweight  Video  Reconnaissance  System 

✓ 

National  Automotive  Center 

✓ 

Near  Term  Digital  Radio 

✓ 

Precision  Lightweight  Global  Positioning  System 
Receiver 

■/ 

Department  of  the  Navy 

Advanced  Deployable  System 

V' 

ARC-210  Very  High  Frequency/ 

Ultra  High  Frequency  Radio 

✓ 

Battle  Group  Passive  Horizon  Extension  System 
Surface  Terminal 

✓ 

Combat  Systems  Engineering 

V 

•/ 

Common  Support  Equipment 

Control  Display  Navigation  Unit 

Fixed  Distributed  System 

✓ 
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Reviewed _ _ _  ■— 


Modified 

Commercial 

Nondeveloomental 

Department  of  the  Navy  (cont’d) 

Ground  Proximity  Warning  System 

V' 

High  Frequency  Radio  Group 

✓ 

Hull,  Mechanical  and  Electrical  Equipment  Data 
Resources  System 

✓ 

Joint  Maritime  Command  Information  System-Afloat 

Joint  Power  Projection/  Real  Time  Support 

•/ 

Medium  Tactical  Vehicle  Remanufacturing  Program 

✓ 

Miniature  Digital  Assigned  Multiple  Access 

New  Attack  Submarine 

✓ 

PlOO  Portable  Firefighting  Group 

Riverine  Assault  Craft 

✓ 

Strategic  Systems  Program 

V 

✓ 

Submarine  Message  Buffer 

V 

Surface  Ship  Torpedo  Defense 

a.  Launched  Expendable  Acoustic  Device 

b.  Multi-Sensor  Torpedo  Recognition  and 

Alertment  Processor 

✓ 

✓ 

Surveillance  Towed  Array  Sensor  System 

✓ 

Surveillance  Towed  Array  Sensor  System 
-  Low  Frequency  Active 

Department  of  the  Air  Force 

C-130J  Aircraft 

•/ 

Commercial  Aircraft  Program 

Military  Products  From  Commercial  Lines 

T-IA  Aircraft 


26 


Appendix  D.  Buying  Organizations’  Lessons 
Learned  Summarized  by  Program  Management 
Element  for  Modified  Commercial  Items 


Program  Definition 

•  Users  should  use  knowledge  of  the  marketplace  to  develop  and  modify  operational 
requirements,  which  would  create  greater  opportunity  for  use  of  commercial  items 
(FAR  10.002  and  11.002[a][2][v]). 

•  Users  should  not  mix  military  and  commercial  requirements  (specifications)  in 
operational  requirements. 

•  Users  should  structure  operational  requirements  so  that  commercial  items  can  meet 
military  manpower  and  personnel  criteria  (FAR  10.002[b]  and  11.002[a][2][ii]). 

•  Users  should  develop  requirements  Aat  take  maximum  adv^tage  of  commercial 
items  with  previous  commercial  testing  (FAR  11.002[a][2][ii]). 

•  Buying  organizations  must  communicate  and  cooperate  early  with  the  user  to 
ensure  flexibility  in  the  system  requirements  and  to  share  knowledge  of  potential 
commercial  items  that  may  be  available  to  meet  requirements  (FAR  11.203). 

•  Buying  organizations  should  get  the  user  to  buy-in  early  on  contract 
requirements.* 


Program  Structure 

Acquisition  Strategy 

Integration: 

•  Buying  organizations  should  perform  risk  analyses  to  ensure  that  the 
Government  has  accounted  for  aU  possible  pitfalls  in  the  acquisition, 
support,  and  life-cycle  management  of  commercially  procured  systems. 

•  The  buying  organizations  should  require  extensive  user  involvement  when 
integrating  commercial  hardware  and  software  specification  products  into 
military  specification  platforms. 

•  The  prime  integrating  contractor  needs  to  be  a  major  stakeholder  in  the 
acquisition.* 

•  When  commercial  suppliers  use  commercial  software,  buying  organizations 
should  obtain  an  understanding  of  the  integrators'  plans  and  proces^s  to 
select,  evaluate,  and  incorporate  commercial  software  products  during 
software  and  system  development. 

♦  Buying  organizations  identified  asterisked  lessons  learned  as  common  for  acquiring  modified  commercial  and 
nondevelopmental  items. 
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Communication: 

•  DoD  needs  to  learn  how  to  act  commercially  to  reap  benefits  from  buying 
commercial  items. 

•  Buying  organizations  need  to  improve  their  understanding  of  commercial 
practices. 

•  Buying  organizations  should  learn  how  to  work  with  contractors' 
businesses  as  usual  methods  of  operations. 

•  The  Defense  Contract  Management  Command  and  the  Defense  Contoct 
Audit  Agency  should  buy-in  as  the  contracting  team  develops  the  price 
analysis  strategy.* 

Market  Research: 

•  Buying  organizations  should  maximize  the  use  of  standard  commercial 
equipment  (FAR  11. 002[a]  [2]). 

•  Buying  organizations  should  perform  market  surveys  to  determine  the 
extent  of  competition  and  the  feasibility  of  using  modified  commercial 
products. 

•  In  determining  the  feasibility  of  using  commercial  items,  buying 
organizations  should  assess  the  total  system  operation  and  support 
effectiveness,  not  just  system  performance. 

•  Buying  organizations  should  explicitly  consider  system-level  concerns 
relating  to  commonality  between  subsystems,  commonality  between 
programs,  compliance  with  open  system  architecture  standards  and  guides, 
and  supportability  in  the  commercial  assessment  and  selection  process. 

•  Buying  organizations  must  ensure  that  commercial  software  products  meet 
known  interface  standards  (FAR  11.002[a][2]). 

•  Buying  organizations  should  use  market  research  to  identify  hardware  with 
a  clear  upgrade  path  and  capability  to  transition  within  the  same  hardware 
and  operating  system  families.* 

•  Buying  organizations  should  use  market  research  to  identify  assemblies  that 
are  readily  adaptable  to  future  modifications  (FAR  10.(X)l[a][3]).* 

•  Buying  organizations  should  use  market  research  to  obfcdn  commercial 
systems,  subsystems,  and  components  from  stable  suppliers 

(FAR  10.001[a][3]). 

•  Buying  organizations  should  base  market  research  decisions  on  fairness, 
competition,  and  ethics  (FAR  1.102[b][3]).* 


*  Buying  organizations  identified  asterisked  lessons  learned  as  common  for  acquiring  modified  commercial  and 
nondevelopmental  items. 
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•  Buying  organizations  should  aggressively  dialogue  with  interested  vendor 
teams  to  determine  the  “best  that  industry  can  offer.  ”♦ 

•  Buying  organizations  should  contact  other  commercial  operators  of  similar 
products  (FAR  10.002[b][2][i]). 

•  Buying  organizations  should  reevaluate  Government  requirements  if  the 
ability  of  only  a  few  sources  to  compete  in  the  aicquisition  is  apparent 
(FAR  10.002).* 

•  Buying  organizations  should  use  state-of-the-practice  rather  than  state-of- 
the-art  hardware  and  software  requirements  to  reduce  program  risk.* 

•  Buying  organizations  should  fully  understand  commercial  product 
procurement  and  testing  information. 

•  Buying  organizations  should  identify  aU  commercial  options  up-ffont  and 
not  leave  the  choice  up  to  the  prime  contractors. 


Budgeting  and  Scheduling: 

•  Buying  organizations  need  to  understand  limited  availability  and  visibility 
into  the  contractor’s  cost  or  pricing  data. * 

•  Buying  organizations  should  thoroughly  understand  commercial  cost 
baselines  and  options. 

•  Buying  organizations  should  plan  for  and  allocate  additional  integration 
time  and  resources  to  resolve  interface  and  performance  problems  during 
system  development  because  interoperability  is  a  key  issue  with 
commercial  products. 

•  Buying  organizations  should  recognize  that  contractors  will  most  likely 
require  additional  time  and  effort  to  incoiporate  new  commercial  software 
product  releases  into  a  system  and  to  verify  system  compatibility. 

•  Buying  organizations  should  plan  for  change  with  technology  refreshment 
when  using  commercial  technology. 

Logistical  Support: 

•  Buying  organizations  should  reco^ze  logistical  support  as  the  highest  risk 
associated  vdth  modified  commercial  and  nondevelopmental  item 
acquisitions.* 

•  Buying  organizations  need  to  verify  that  commercial  supplier  parts  support 
is  longer  than  the  anticipated  part  life-cycle. 


*  Buying  organizations  identified  asterisked  lessons  learned  as  common  for  acquiring  modified  commercial  and 
nondevelopmental  items. 
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•  Buying  organizations  should  consider  requiring  that  a  third  party  hold 
technical  data  in  escrow  for  Government  use  ^r  commercial  supplier 
support  ends  if  the  product  life-cycles  extend  beyond  commercial  supplier 
support. 

•  DoD  needs  to  develop  training  programs  to  educate  personnel  in  buying 
and  using  “best  commercial  practices”  to  ensure  that  logistical  support  is 
available  when  items  are  fielded. 

•  Buying  organizations  should  thoroughly  consider  reprocurement  needs  in 
the  contract  source-selection  process.* 

Test  and  Evaluation 

•  Buying  organizations  should  consider  requiring  the  performance  of 
operational  testing  and  evaluation  as  part  of  the  source-selection  process.* 

•  Buying  organizations  should  use  conformance-testing  of  commercial 
products  in  the  source-selection  process  to  provide  assurance  that  products 
function  properly  in  a  specific  integrated  system  configuration  (the  larger 
and  more  complex  the  system,  the  higher  the  risk). 

•  Buying  organizations  should  develop  a  sensible  test  program  using  previous 
manufacturer  and  Government  test  results.* 

•  Buying  organizations  should  tailor  testing  to  proven  technology  to  cut  costs 
and  to  accelerate  the  program  schedule.* 

•  Buying  organizations  should  tailor  testing  to  address  program  risk  areas.* 

•  Buying  organizations  should  organize  testing  to  maximize  the  use  of  the 
tests.* 

•  Buying  organizations  should  plan  the  conduct  of  operational  testing  as 
early  as  possible.* 

•  Buying  organizations  should  plan  adequate  time  for  debugging  and  design 
updating  of  systems  provided  as  Government-furnished  equipment.* 

•  Buying  organizations  must  recognize  that  early  test  and  evaluation  of 
commercial  components  is  important  when  commercial  suppliers  are 
modifying  a  commercial  system. 

•  To  be  successful,  test  organizations  need  a  well  defined  requirements 
document  and  a  follow-up  test  and  evaluation  master  plan.* 

•  Buying  organizations  should  ensure  that  performance  pass  or  fail  criteria 
are  clearly  specified  in  the  contract.* 


*  Buying  organizations  identified  asterisked  lessons  learned  as  common  for  acquiring  modified  commercial  and 
nondevdopmental  items. 
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Appendix  D.  Buying  Organizations’  Lessons  Learned  Summarized  by  ftogram 
_ Management  Element  for  Modified  Commercial  Items 


•  Buying  organizations  should  thoroughly  analyze  known  deficiencies  of 
commercM  equipment  before  contract  award. 

•  Buying  organizations  should  share  test  plans  and  results  with  contractors.* 

•  Test  organizations  should  maintain  on-site  representation  during  test 
execution.* 

•  Buying  organizations  should  ensure  that  newly  delivered  equipment  is  initially 
single-sited  for  testing  purposes  to  avoid  spreading  any  undetected 
problems.* 

•  Test  organizations  should  test  software  acquisitions  in  each  fielded 
configuration  of  an  item  to  ensure  that  no  operability  problems  are  related  to 
using  different  item  suppliers.* 

•  Buying  organizations  must  adequately  test  commercial  products  to  ensure 
that  the  products  live  up  to  their  specifications  because  commercial 
specifications  tend  to  be  optimistic. 


Program  Design 

Engineering  and  Manufacturing 

Reliability  and  Maintainability: 

•  Contracting  officers  should  require  contractors  to  fully  disclose  item 
reliability  and  maintainability  data.* 

Design  for  Manufacturing  and  Production: 

•  Buying  organizations  must  recognize  that  modified  commercial  items 
require  both  fabrication  and  integration  as  compared  to  a  commercial 
off-the-shelf  product. 

•  Buying  organizations  need  to  acquire  systems  that  use  open  systems 
architecture  to  negate  the  need  for  commercial  product  data  to  avoid  using 
single-source  products  or  product  features  that  are  unique  to  a  single 
manufacturer. 

•  Buying  organizations  should  avoid  modifying  commercial  products,  if 
possible,  to  satisfy  stated  military  requirements. 

•  When  selecting  circuit  card  assemblies  for  computers,  buying  organizations 
should  ensure  that  contractors  select  items  that  are  easily  upgradable 
because  the  marketplace  rather  then  the  Government  drives  the  life-cycle 
costs  of  the  equipment. 

•  System  integrators  should  consider  prototyping  as  means  to  identify 
interoperability  problems  related  to  the  use  of  different  commercial 
suppliers  for  critical  commercial  products. 


*  Buying  organizations  identified  asterisked  lessons  learned  as  conunon  for  acquiring  modified  commercial  and 
nondevelopmental  items. 
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Appendix  D.  Buying  Organizations’  Lessons  l^med  Sununarized  by  Program 
Management  Element  for  Modified  Commercial  Items  _ 


Logistics  and  Other  Infrastructure 

Maintenance  Support: 

•  Buying  organizations  should  recognize  that  long-lead  items  such  as 
modems,  multiplexers,  reflectors,  and  high  performance  amplifiers  are  too 
expensive  to  stock  on  shelves.* 

•  Buying  organizations  should  assess  whether  contractor  logistical  support 
systems  are  capable  of  processing  large  volumes  of  repairs  within  limited 
times.* 

•  Buying  organizations  should  place  more  reliance  on  use  of  proven 
contractor  logistical  support  networks. 

•  For  commercial  acquisitions,  buying  organizations  should  accept  and  use 
the  support  system  that  is  already  in  place  for  commercial  customers 
(FAR  10.002[b][l][vi]). 

•  Contractors  are  reluctant  to  disclose  expected  failure  rates  for  parts 
provisioning  after  warranties  expire. 

•  Buying  organizations  should  recognize  that  parts  obtain^  from  different 
commercial  suppliers  may  not  function  the  same  as  original  equipment 
manufacturers’  parts. 

•  Buying  organizations  should  not  abuse  contractor  support  networks  for 
warranty  repairs.* 

Warranties  and  Data  Rights: 

•  Buying  organizations  should  ensure  that  warranties  are  enforceable 
(FAR  12.404).* 

•  Buying  organizations  should  understand  that  tracking  warranty  rep^s  and 
recalls  can  be  burdensome.* 

•  Buying  organizations  should  understand  that  long-term  contract  warranties 
may  mean  that  obtaining  technical  data  rights  is  unnecessary  (FAR  12.211).* 

•  Buying  organizations  should  thoroughly  consider  data  rights  requirements 
for  commercial  equipment  (FAR  12.211). 

•  Commercial  suppliers  will  generally  not  release  proprietary  rights  to 
technical  data  that  are  needed  to  establish  organic  repair  (FAR  12.21 1  and 
Defense  Federal  Acquisition  Regulation  Supplement  227.7102). 

•  Commercial  product  documentation  is  sometimes  incomplete. 


*  Buying  organizations  identified  asterisked  lessons  learned  as  common  for  acquiring  modified  commercial  and 
nondevelopmental  items. 
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Appendix  D.  Buying  Organizations*  Lessons  Learned  Sununarized  by  ftogram 
_ Management  Element  for  Modified  Conunercial  Items 


•  Buying  organizations  should  avoid  customizing  technical  data  requirements 
and  should  take  full  advantage  of  available  commercial  technical  data 
(FAR  12.211  and  Defense  Federal  Acquisition  Regulation 
Supplement  227.7102). 

•  Contracting  officers  should  consider  including  contract  option(s)  for  parts 
and  techni^  data  buyouts  to  support  future  user  logistics  requirements. 

•  Buying  organizations  should  purchase  the  necessary  technical  data  rights 
for  electrical  interfaces,  protocols,  and  ijerformance  parameters  of  critical 
circuit  card  subcomponents  when  obtaining  open  system  architecture 
components  to  preserve  the  opportunity  to  accomplish  later  growth  options 
and  engineering  change  proposals  in  an  inexpensive  manner  (Defense 
Federal  Acquisition  Regulation  Supplement  227.7102). 

Configuration  Management: 

•  Buying  organizations  should  recognize  that  commercial  technology 
configuration  management  requires  line  replaceable  unit  management 
rather  than  piece  part  accountability  and  control. 

•  Buying  organizations  should  understand  that  configuration  tracking  and 
material  support  can  become  difficult  when  commercial  suppliers  make 
changes  to  equipment  unilaterally. 

•  Buying  organizations  should  communicate  item  configuration  changes  to 
the  user  during  the  commercial  item  life-cycle. 

•  Buying  organizations  should  maintain  a  software  configuration  item 
baseline  if  systems  will  be  placed  on  multiple  platforms.* 

Documentation: 

•  Buying  organizations  should  require  that  user  system  manuals  reference 
component  and  subsystem  commercial  manuals  and  discuss  integration  of 
commercial  equipment. 

•  Buying  organizations  can  reduce  required  software  documentation  and  the 
number  of  program  reviews  by  using  commercial  software  (FAR  12.212). 

Parts  Provisioning: 

•  Buying  organizations  should  test  reprocured  items  to  ensure  form,  fit,  and 
function  compatibility  because  commercial  suppliers  make  technical 
changes  without  changing  parts  numbers. 


*  Buying  organizations  identified  asterisked  lessons  learned  as  common  for  acquiring  modified  commercial  and 
nondevelopmental  items. 
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Appendix  D.  Buying  Organizations’  Lessons  Learned  Sununarized  by  Program 
Manflffftmfint  Element  for  Modified  Commercial  Items  _ _ _ 


Contracting 

•  Contracting  officers  should  clearly  define  item  requirements  before  contract  award 
(FAR  12.202[b])  * 

•  Contracting  officers  should  understand  that  open  system  architecture  electronic 
components  are  not  always  plug  and  play  and  that  integrators  and  subcontractors’ 
products  may  not  always  be  compatible. 

•  Industry  input  on  product  descriptions  and  statements  of  work  helps  clarify 
technical  aspects  and  helps  reveal  alternative  ways  to  meet  requirements 
(FAR  11.002[c]). 

•  Buying  organizations  should  ensure  that  product  descriptions  reflect  requirements 
flexibility  by  defining  functions  to  be  performed  and  by  using  such  terms  as 
acceptable  ranges,  targets,  or  desired  and  required  values  rather  than  exact  values 
(FAR  11.002[a][2]).* 

•  Contracting  officers  should  prepare  requests  for  proposal  in  industry  format 
(FAR  12.302[a]). 

•  Contracting  officers  should  start  out  with  a  commercial  request  for  proposal  and 
model  the  contract  accordingly  (FAR  12.302[a]). 

•  Contracting  officers  should  not  include  burdensome  proposal  instructions  in  requests 
for  proposal  (FAR  12.301[b][l]).* 

•  Contracting  officers  should  detdl  the  repair  and  return  concept  down  to  the 
component  level  in  requests  for  proposjfl.* 

•  Contracting  officers  should  pay  close  attention  to  product  support  in  requests  fijr 
proposal.* 

•  Contracting  officers  should  require  the  conduct  of  proposal  or  preaward  testing  so 
that  equipment  can  be  tested  for  unique  military  requirements  and  checked  for 
interfaces  with  existing  systems. 

•  Contracting  officers  should  set  up  vendor  demonstrations  to  determine  whether 
actual  product  performance  differs  from  advertised  performance. 

•  Contracting  officers  should  establish  that  only  the  minimum  essential  features  are 
being  demonstrated  or  else  extend  the  time  period  to  permit  full  development  of  a 
bid  sample  to  increase  competition  on  Government  contracts. 

•  Contracting  officers  should  consider  the  use  of  operational  demonstrations  during 
source  selection. 

•  Contracting  officers  should  require  verification  testing  of  contractor-furnished 
product  performance  data. 


*  Buying  organizations  identified  asterisked  lessons  learned  as  common  for  acquiring  modified  commercial  and 
nondevelopmental  items. 


34 


Appendix  D.  Buying  Organizations*  Lessons  Learned  Sununarized  by  Program 
_  Management  Element  for  Modified  Commercial  Items 


•  Contracting  officers  should  compare  the  quality  of  the  product  and  future  service 
requirements  against  the  contractors’  past  performance  to  meet  current  needs 
(FAR  12.206). 

•  Contracting  officers  should  use  innovative  cost  or  pricing  analysis  techniques.  * 

•  Contracting  officers  will  find  it  difficult  to  negotiate  with  a  sole-source  provider 
when  cost  or  pricing  data  are  not  required  even  if  the  item  has  a  limited 
commercial  sdes  history. 

•  Contracting  officers  should  include  most  favored  commercial  customer  provisions 
in  contracts  when  such  clauses  are  consistent  with  standard  commercial  practice 
(FAR  12.302[a]). 

•  Contracting  officers  should  put  flexibility  in  contracts  to  accommodate  variations 
to  system  network  architectures.* 

•  Military  commanders  must  fully  understand  and  appreciate  commercial  practice 
contract  terms. 

•  Contracting  officers  should  clearly  define  the  contract  administration 
organization’s  role  in  product  quality  assurance  in  contracts  within  the  limitations 
of  FAR  12.208.* 

•  Contracting  officers  should  consider  including  contract  provisions  requiring 
commercial  suppliers  to  update  technical  documentation  to  reflet  configuration 
management  changes  when  such  requirements  are  consistent  with  standard 
commercial  practice  (FAR  12.302[a]). 

•  Contracting  officers  should  write  concise  contract  data  requirements  lists  when 
purchasing  data  in  accordance  with  FAR  12.211  and  Defense  Federal  Acquisition 
Regulation  Supplement  227.7102.* 

•  Contracting  officers  should  contract  for  long-term  warranties  when  past  contractor 
performance  indicates  program  risk.* 

•  Contracting  officers  should  time  procurement  actions  as  closely  as  possible  to  the 
requirement  time  to  preclude  parts  obsolescence  and  replacement.* 

•  Contracting  officers  should  concurrently  award  procurement  and  logistical  support 
contracts  to  reduce  program  costs.* 


*  Buying  organizations  identified  asterisked  lessons  learned  as  common  for  acquiring  modified  commercial  and 
nondevelopmental  items. 
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Appendix  E.  Buying  Organizations’  Lessons 
Learned  Summarized  by  Program  Management 
Element  for  Nondevelopmental  Items 

Program  Definition 

•  Buying  organizations  should  get  the  user  to  buy-in  early  on  contract  requirements.  * 


Program  Structure 

Acquisition  Strategy 

Integration: 

•  Buying  organizations  should  recognize  that  integration  of 
nondevelopmental  items  is  very  difficult  and  should  treat  it  as  a  design 
problem  and  not  a  simple  production  problem. 

•  The  prime  integrating  contractor  needs  to  be  a  major  stakeholder  in  the 
acquisition,* 

Conununication: 

•  The  Defense  Contract  Management  Command  and  the  Defense  Contoct 
Audit  Agency  should  buy-in  as  the  contracting  team  develops  the  price 
analysis  strategy.* 

Market  Research: 

•  Buying  organizations  should  use  market  research  to  identify  hardware  with 
a  clear  upgrade  path  and  capability  to  transition  within  the  same  hardware 
and  operating  system  families.* 

•  Buying  organizations  should  use  market  research  to  identify  assemblies  that 
are  readily  adaptable  to  future  modifications  (FAR  10.001[a][3]).* 

•  Buying  organizations  should  base  market  research  decisions  on  fairness, 
competition,  and  ethics  (FAR  1.102[b][3]).* 

•  Buying  organizations  should  aggressively  dialogue  with  interested  vendor 
teams  to  determine  the  “best  that  industry  can  offer.”* 

•  Buying  organizations  should  reevaluate  Government  requirements  if  the 
ability  of  only  a  few  sources  to  compete  in  the  acquisition  is  apparent 
(FAR  10.002).  * 

•  Buying  organizations  should  use  state-of-the-practice  rather  than  state-of- 

the-art  hardware  and  software  requirements  to  reduce  program  risk.* 

*  Buying  organizations  identified  asterisked  lessons  learned  as  common  for  acquiring  modified  commercial  and 
nondevelopmental  items. 
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Appendix  E.  Buying  Organizations’  Lessons  Learned  Sununarized  by  Program 
_ Management  Element  for  Nondevelopmental  Items 


Budgeting  and  Scheduling: 

•  Buying  organizations  need  to  understand  limited  availability  and  visibility 
into  the  contractor’s  cost  or  pricing  data.* 

Logistical  Support: 

•  Buying  organizations  should  recognize  logistical  support  as  the  highest  risk 
associated  with  modified  commercial  and  nondevelopmental  item 
acquisitions.* 

•  Buying  organizations  should  thoroughly  consider  reprocurement  needs  in 
the  contract  source-selection  process.* 

Test  and  Evaluation 

•  Buying  organizations  should  consider  requiring  the  performance  of 
operational  testing  and  evaluation  as  part  of  the  source-selection  process.* 

•  Buying  organizations  should  develop  a  sensible  test  program  using  previous 
manufacturer  and  Government  test  results.* 

•  Buying  organizations  should  tailor  testing  to  proven  technology  to  cut  costs 
and  to  accelerate  the  program  schedule.* 

•  Buying  organizations  should  tailor  testing  to  address  program  risk  areas.* 

•  Buying  organizations  should  organize  testing  to  maximize  the  use  of  the 
tests.* 

•  Buying  organizations  should  plan  the  conduct  of  operational  testing  as 
early  as  possible.* 

•  Buying  organizations  should  plan  adequate  time  for  debugging  and  design 
updating  of  systems  provided  as  Government-furnished  equipment.* 

•  To  be  successful,  test  organizations  need  a  well  defined  requirements 
document  and  a  follow-up  test  and  evaluation  master  plan.* 

•  Buying  organizations  should  ensure  that  poformance  pass  or  fail  criteria 
are  clearly  specified  in  the  contract.* 

•  Buying  organizations  should  share  test  plans  and  results  with  contractors.* 

•  Test  organizations  should  maintain  on-site  representation  during  test 
execution.* 

•  Buying  organizations  should  ensure  that  newly  delivered  equipment  is 
initially  single-sited  for  testing  purposes  to  avoid  spreading  any  undetected 
problems.* 


*  Buying  organizations  identified  asterisked  lessons  learned  as  common  for  acquiring  modified  commercial  and 
nondevelopmental  items. 
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Appendix  E.  Buying  Organizations’  Lessons  Learned  Sununarized  by  Program 
Management  Element  for  Nondevelopmental  Items _ _ _ 


•  Test  organizations  should  test  software  acquisitions  in  each  fielded 

configuration  of  an  item  to  ensure  that  no  operability  problems  are  related 
to  using  different  item  suppliers.* 


Program  Design 

Engineering  and  Manufacturing 

Reliability  and  Maintainability: 

•  Contracting  officers  should  require  contractors  to  fully  disclose  item 
reliability  and  maintainability  data.* 

•  Test  organizations  should  validate  the  reported  reliability  of 
nondevelopmental  item  components  and  test  them  thoroughly  in  new 
operational  environments. 

Logistics  and  Other  Infrastructure 

Maintenance  Support: 

•  Buying  organizations  should  carefully  evaluate  options  to  detem^e  which 
is  more  cost-effective,  to  replace  or  to  repair  nondevelopmental  items, 
when  developing  the  acquisition  strategy. 

•  Buying  organizations  should  recognize  that  long-lead  items  such  as 
modems,  multiplexers,  reflectors,  and  high  performance  amplifiers  are  too 
expensive  to  stock  on  shelves.* 

•  Buying  organizations  should  assess  whether  contractor  logistical  support 
systems  are  capable  of  processing  large  volumes  of  repairs  within  limited 
times.* 

•  Buying  organizations  should  not  abuse  contractor  support  networks  for 
warranty  repairs.* 

Warranties  and  Data  Rights: 

•  Buying  organizations  should  ensure  that  warranties  are  enforceable 
(FAR  12.404).* 

•  Buying  organizations  should  understand  that  tracking  warranty  rq)airs  and 
recalls  can  be  burdensome.* 

•  Buying  organizations  should  understand  that  long-term  contract  warranties 
may  mean  that  obtaining  technical  data  rights  is  unnecessary 

(FAR  12.211).* 

Configuration  Management: 

•  Buying  organizations  should  maintain  a  software  configuration  item 
baseline  if  systems  will  be  placed  on  multiple  platforms.* 


*  Buying  organizations  identified  asterisked  lessons  learned  as  common  for  acquiring  modified  commercial  and 
nondevelopmental  items. 
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Appendix  E.  Buying  Organizations’  Lessons  Learned  Sununarized  by  Program 
_ _ Management.  Element  for  Nondevelopmental  Items 


Contracting 

•  Contracting  officers  should  clearly  define  item  requirements  before  contract  award 
(FAR  12.202[b]).* 

•  Buying  organizations  should  ensure  that  product  descriptions  reflect  requirements 
flexibility  by  defining  functions  to  be  performed  and  by  using  such  terms  as 
acceptable  ranges,  targets,  or  desired  and  required  values  rather  than  exact  values 
(FAR  11.002[a][2]).* 

•  Contracting  officers  should  not  include  burdensome  proposal  instructions  in 
requests  for  proposal  (FAR  12.3Ql[b][l]). 

•  Contracting  officers  should  detail  the  repair  and  return  concept  down  to  the 
component  level  in  requests  for  proposal.* 

•  Contracting  officers  should  pay  close  attention  to  product  support  in  requests  for 
proposal,* 

•  Contracting  officers  should  use  innovative  cost  or  pricing  analysis  techniques.* 

•  Contracting  officers  should  put  flexibility  in  contracts  to  accommodate  variations 
to  system  network  architectures.* 

•  Contracting  officers  should  clearly  define  the  contract  administration 

organization’s  role  in  product  quality  assurance  in  contracts  within  the  limitations 
of  FAR  12.208.* 

•  Contracting  officers  should  write  concise  contract  data  requirements  lists  when 
purchasing  data  in  accordance  with  FAR  12.21 1  and  Defense  Federal  Acquisition 
Regulation  Supplement  227.7102.* 

•  Contracting  officers  should  contract  for  long-term  warranties  when  past  contractor 
performance  indicates  program  risk.* 

•  Contracting  officers  should  time  procurement  actions  as  closely  as  possible  to  the 
requirement  time  to  preclude  parts  obsolescence  and  replacement.* 

•  Contracting  officers  should  concurrently  award  procurement  and  logistical  support 
contracts  to  reduce  program  costs.* 


*  Buying  organizations  identified  asterisked  lessons  learned  as  common  for  acquiring  modified  commercial  and 
nondevelopmental  items. 
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Appendix  F.  Categories  of  Acquisition  Programs 


DoD  divides  acquisition  programs  into  categories,  generally  by  size  and 
complexity,  to  assist  in  decisionmaking  and  implementation  of  statutory 
requirements.  The  acquisition  categories  are  as  follows. 


Acquisition  Category  I 


Acquisition  Category  I  programs  are  major  Defense  acquisition  programs.  A 
major  Defense  acquisition  program  is  defined  as  a  program  estimated  to  require 
eventual  expenditure  for  research,  development,  test,  and  evaluation  of  more 
than  $355  million  (FY  1996  constant  dollars)  or  procurement  of  more  than 
$2,135  billion  (FY  1996  constant  dollars),  or  those  designated  to  be  Acquisition 
Category  I  programs. 


Acquisition  Category  II 

Acquisition  Category  n  programs  are  defined  as  those  acquisition  programs  that 
do  not  meet  the  criteria  for  an  Acquisition  Category  I  program,  but  do  meet  the 
criteria  for  a  major  system.  A  major  system  is  defined  as  a  program  estimated 
to  require  eventual  expenditure  for  research,  development,  test,  and  evaluation 
of  more  than  $75  million  in  FY  1980  constant  dollars  (approximately 
$140  million  in  FY  1996  constant  dollars),  or  for  procurement  of  more  than 
$300  million  in  FY  1980  constant  dollars  (approximately  $645  million  in 
FY  1996  constant  dollars),  or  those  designated  to  be  Acquisition  Category  n 
programs. 


Acquisition  Category  HI  (all  other  acquisition  programs) 


Acquisition  Category  m  programs  are  defined  as  those  acquisition  programs 
that  do  not  meet  the  criteria  for  an  Acquisition  Category  I  or  an  Acquisition 
Category  n  program. 
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Appendix  G.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 
Deputy  Under  Secretary  of  Defense  (Acquisition  Reform) 

Deputy  Under  Secretary  of  Defense  (Industrial  Affairs  and  Installations) 
Director,  Defense  Logistics  Studies  Information  Exchange 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

General  Council,  Department  of  Defense 
Assistant  Secretory  of  Defense  (Public  Affairs) 


Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Research,  Development  and  Acquisition) 
Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 


Assistant  Secretory  of  the  Navy  (Financial  Management  and  Compttoller) 
Assistant  Secretary  of  the  Navy  (Research,  Development  and  Acquisition) 
Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Assistant  Secretory  of  the  Air  Force  (Acquisition) 

Assistant  Secretary  of  the  Air  Force  financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 
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Other  Defense  Organizations 

Director,  Ballistic  Missile  Defense  Organiration 
Director,  Defense  Advanced  Research  Projects  Agency 
Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Logistics  Agency 
Director,  Defense  Special  Weapons  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 


Non-Defense  Organizations 

Office  of  Management  and  Budget  .  ,  , 

Technical  Information  Center,  National  Security  and  International  Affairs  Division, 
General  Accounting  Office 

Chairman  and  ranking  minority  member  of  each  of  the  following  congressional 
committees  and  subcommittees: 

Senate  Committee  on  Appropriations 
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